Support multi images for vlm benchmarking in samples and llm_bench #2197
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
category: llm_bench
Wovchena
reviewed
May 12, 2025
wgzintel
changed the title
…genai into guozhong/support_multi_files_for_vlm_test
…ithub.com/wgzintel/openvino.genai into guozhong/support_multi_files_for_vlm_test
…genai into guozhong/support_multi_files_for_vlm_test
Wovchena
requested changes
Jun 17, 2025
…genai into guozhong/support_multi_files_for_vlm_test
Wovchena
reviewed
Jun 17, 2025
sbalandi
reviewed
Jun 17, 2025
sbalandi
reviewed
Jun 17, 2025
Co-authored-by: Vladimir Zlobin <[email protected]>
Co-authored-by: Sofya Balandina <[email protected]>
Co-authored-by: Sofya Balandina <[email protected]>
sbalandi
reviewed
Jun 18, 2025
…genai into guozhong/support_multi_files_for_vlm_test
…ithub.com/wgzintel/openvino.genai into guozhong/support_multi_files_for_vlm_test
Wovchena
approved these changes
Jun 19, 2025
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for handling multiple images in VL benchmarking and enhances the prompt configuration by allowing prompt file input across Python and C++ samples.
- Updated image loading logic in both Python and C++ to handle directories of images.
- Modified prompt handling to allow reading from files and set default prompts accordingly.
- Documentation and build script updates to reflect the new user options.
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/llm_bench/task/visual_language_generation.py | Uses directory-checks to load multiple images and logs the image count. |
| samples/python/visual_language_chat/benchmark_vlm.py | Introduces prompt file support and updates image loading to handle multiple images. |
| samples/python/visual_language_chat/README.md | Updates command line options documentation for prompt and image inputs. |
| samples/python/text_generation/benchmark_genai.py | Modifies prompt handling to support file input and consistent list formatting for tokenization. |
| samples/cpp/visual_language_chat/benchmark_vlm.cpp | Updates image loading to support multiple images and integrates prompt file reading. |
| samples/cpp/visual_language_chat/README.md | Updates documentation to reflect prompt file and multi-image support. |
| samples/cpp/visual_language_chat/CMakeLists.txt | Adjusts the build configuration to include the new source for prompt file reading. |
| samples/cpp/text_generation/read_prompt_from_file.{h,cpp} | Introduces utility functions to read prompts from a file. |
| samples/cpp/text_generation/benchmark_genai.cpp | Similar prompt handling updates with multi-image and prompt file support. |
| samples/cpp/text_generation/README.md | Reflects updated command line options for prompt configuration. |
| samples/cpp/text_generation/CMakeLists.txt | Adjusts the build configuration to include the new prompt file reader utility. |
sbalandi
approved these changes
Jun 19, 2025
github-merge-queue
bot
removed this pull request from the merge queue due to a conflict with the base branch
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.